Musar על אבות 5:4

And it is stated in Sifrei, and also in Arachin 15a that our forefathers tried the L-rd ten times, and their decree was sealed only for the sin of lashon hara alone, as it is written (Bamidbar 14:22): "And they tried Me this ten times, and they did not heed My voice," and (Malachi 2:17): "You have wearied the L-rd with your words."